Computer code test script generating tool using visual inputs

ABSTRACT

A tool includes an interface, a memory, a conversion engine, an identifier tool, and a script engine. The interface communicatively couples the tool to a server. The tool obtains a plurality of visual inputs from a computer program specification document. The memory stores the plurality of visual inputs and a set of known computer code elements. Each respective element of the set of known computer code elements includes predetermined testing criteria far testing computer code that includes the respective element. The conversion engine generates a plurality of textual objects from the plurality of visual inputs. The identifier determines whether each respective textual object matches a respective element of the set of known computer code elements. If a match is found, the identifier tool associates the predetermined testing criteria of the respective element to the respective textual object. The script engine generates a test script using the predetermined testing criteria.

TECHNICAL FIELD

The present disclosure relates generally to testing computer code, andmore specifically to generating test scripts for testing computer code.

BACKGROUND

One of the technical challenges associated with testing or debuggingcomputer code is writing accurate test scripts that cheek computerprograms for adherence to design specifications, Existing solutionstypically require the completion of a developed computer program beforea test script may be generated. Furthermore, conventional techniquesalso require the correlation of the developed computer program code tothe design specifications for the computer program. Accordingly, currentsolutions typically require additional computer processing and memoryresources associated with writing additional test scripts and checkingthe written test scripts for compliance with subsequent versions of thecomputer program, these additional resources may be expended even if thesubsequent version of the developed computer program follows the samedesign specification. Existing techniques are inadequate and resourceintensive. For example, requiring the completed computer program towrite the test scripts may introduce errors and inaccuracies from thecomputer program code into the test scripts, thereby increasing theamount of processing resources used to test the computer code, e.g., byhaving to run subsequent test scripts that address the incompleteness ofthe earlier test script. Additionally, inadequate test scripts, such asthose that are inaccurate or incomplete, hamstrings the ability to testcomputer programs, thereby increasing the chance of system crashing bugsand security vulnerabilities to appear in the end-user versions of thecomputer program code. Each of these issues increases the amount ofcomputer resources used to run the computer program, e.g., by requiringrestarting/re-running the program after a crash, by introducing memoryleaks when running the program, or allowing, unauthorized programs torun on the system.

SUMMARY

One of the technical challenges associated with testing or debuggingcomputer code is writing accurate test scripts that check computerprograms for adherence to design specifications. Current solutionsresult in unnecessary computer processing and memory resources to beused in the process of generating and using test scripts. For example,test scripts based on the developed computer program code may result inincomplete or inaccurate tests scripts. This may result from theconversion of the design specifications for the computer program tofunctional requirements and the parsing or interpretation of thedeveloped computer code. This process is prone to errors and may furtherresult in test scripts that are only usable on that particular versionof the developed computer program code. For example, subsequent versionsof the computer program may use different styles or implementations ofthe functional elements defined in the specification document. In thiscase, a new test script may have to be used, requiring additionalprocessing resources to generate the test script, run the test scriptagainst the new version, and maintain the correlation between the testscripts and the applicable program versions. With a limited amount ofresources available, the factors described above may result insubstandard code testing and lead to a higher chance for the developmentof non-functioning or insecure computer programs as well as a slowerdevelopment-testing cycle.

The systems and methods described herein provide several technicaladvantages that ensure complete and efficient testing of computer code.In particular, a test script generating tool is provided that generatestest scripts based on visual inputs from a computer programspecification document. The test script generating tool allows for thegeneration of test scripts even before the computer code is written.This provides immediate feedback on his computer code once it is checkedinto the completed computer code repository, thereby decreasing the timebefore bugs or compliance issues are addressed, thereby limiting thewasted resources associated with running buggy or non-compliant computercode. As another example, in certain embodiments, the test scriptgenerating tool incorporates one or more machine learning techniques toidentify elements from the visual inputs of the computer programspecification document. In certain embodiments, the test scriptgenerating tool uses Bayesian analysis to determine the type andcharacteristics of an element in the visual input. Further, in someembodiments, the test script generating tool uses keyword-basedsentimental classification to identify an element in the visual inputbased on text identified from the visual input. The identified elementsmay be used by the test script generating tool to generate a test scriptto test those elements and associated functions and enhance thedetection techniques utilizing machine learning. In this manner, atrained test script generating tool may generate accurate test scriptswhile limiting the amount of resources used to obtaining externalinformation.

Another challenge in testing computer program code is the lack offlexibility in testing the computer code. As mentioned above, typicalsolutions require to the identification of the elements of the developedcomputer program code to be tested and interpreting how to apply thedesign constraints and requirements from the specification document, Asa result, the generated test scripts may only apply to the specificversion of the developed computer program code used, in generating thetest script. Certain systems and methods disclosed herein allow forfine-tuning of the test script using the test script generating tool,For example, in certain embodiments, the test script generating tool maybe communicatively coupled to a user interface, through which a user maycustomize the test script, e.g., by selecting certain identifiedfunctions or elements to test or not test. Based on this user input, thetest script generating tool may generate a test script that is tailoredfor the testing desired for the computer program. Furthermore, userinput may be used by the test script generating tool to enhance itsmachine learning engines. For example, the user may add additionalcontextual information through with which the test script generatingtool may identify the elements based on the visual inputs from thecomputer code specifications document.

Certain embodiments of the present disclosure may include some, all, ornone of these advantages. These advantages and other features will bemore clearly understood from the following detailed description taken inconjunction with the accompanying drawings and claims.

According to an embodiment, a test script generating tool is providedfor generating a test script for use in an automation framework, Thetest script generating tool includes an interface, a memory, aconversion engine, an identifier tool, and a script engine. Theinterface communicatively couples the test script generating tool to aserver. The test script generating tool accesses the server via theinterface to obtain a plurality of visual inputs from a computer programspecification document. Each visual input represents one or morefunctional requirements of the computer program specification document.The memory stores the plurality of visual inputs and a set of knowncomputer code elements. Each respective element of the set of knowncomputer code elements includes predetermined testing criteria fortesting computer code that includes the respective element. Theconversion engine accesses the plurality of visual inputs from thememory. The conversion engine further generates a plurality of textualobjects from the plurality of visual inputs using one or moreprocessors. The plurality of textual objects is associated with the oneor more functional requirements of the computer program specificationdocument. The identifier tool accesses the set of known computer codeelements from the memory. The identifier tool further compares theplurality of textual objects to the set of known computer code elementsto determine, for each respective textual object of the plurality oftextual objects, whether the respective textual object matches arespective element of the set of known computer code elements. If therespective textual object matches a respective element of the set ofknown computer code elements, the identifier tool associates thepredetermined testing criteria of the respective element to therespective textual object. The script engine generates a test scriptusing the predetermined testing criteria associated with the pluralityof the textual objects. The script engine further transmits the testscript to a test script repository server. The automation framework mayaccess the test script from the test script repository server.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure, reference is nowmade to the following brief description, taken in connection with theaccompanying drawings and detailed description, wherein like referencenumerals represent like parts.

FIG. 1 is a schematic diagram of an embodiment of a code testing systemfor generating test scripts and testing computer programs using thegenerated test scripts, according to particular embodiments;

FIG. 2 illustrates example visual input and textual objects used ingenerating a test script, according to particular embodiments;

FIG. 3 is a flowchart of an embodiment of an example method forgenerating a test script, according to particular embodiments; and

FIG. 4 is a schematic diagram of an example test script generating toolfor generating test scripts using visual inputs, according to particularembodiments.

DETAILED DESCRIPTION

The systems and methods disclosed herein provide several technicaladvantages in testing computer programs. Generating test scripts basedon visual inputs from a computer program specification document is anunconventional technique that allows test scripts to be generated beforea developer finishes a completed version of the computer programaccording to the specification document. Existing techniquesconventionally require a test script writer to write a test script basedon the completed computer program and his understanding of thespecification document. Generating test scripts using a test scriptgenerating tool using visual inputs from the specification documentrequires may produce more robust, e.g., complete, and accurate testscripts that save processing resources by catching a high percentage ofbugs in the computer programs before sent to end-users. Furthermore, thedisclosed techniques may also shorten development-testing cycle, therebyreducing the amount of computer resources used in stopping and startingdevelopment due to delays in writing and running test scripts.

FIG. 1 shows an example of a code testing system for generating testscripts using visual inputs and testing computer programs. FIG. 2illustrates example visual inputs and example resulting textual objectsthat may be used by a script generating tool. FIG. 3 is an exampleprocess for generating a test script using visual inputs. FIG. 4 is anexample of an implementation of a test script generating tool forgenerating test scripts.

FIG. 1 is a schematic. diagram of an example embodiment of code testingsystem 100. System 100 may be configured to take visual inputs 106 froma specification document 105, generate one or more test scripts 126using a test script generating tool 110, and use generated testscript(s) 126 with an automation framework 140 to test computer programcode 146 developed using specification document 105. In certainembodiments, code testing system 100 includes at least test scriptgenerating tool 110, a test script repository 130 and an automationframework 140, and a code repository 145. Specification document 105 maybe parsed or otherwise processed to generate visual inputs 106. Visualinputs 106 from specification document 105 may indicate the functionalor other requirements of the computer program to be developed based onspecification document 105. For example, visual inputs 106 may includeportions of a flow diagram describing how to implement functional blocksof the computer program or certain style or user design specifications.As a result, visual inputs 106 may represent the functional requirementsof the computer program code to be developed based on specificationdocument.

In system 100, visual inputs 106 from specification document 105 may betransmitted to or otherwise obtained by test script generating tool 110.Test script generating tool 110 may use visual inputs 106 to generate atest script that may be used to test computer programs developed usingspecification document 105. In certain embodiments, test scriptgenerating tool 110 includes conversion engine 115, identifier tool 120,and script engine 125. Test script generating tool 110 may useconversion engine 115 to convert each of visual inputs 106 into textualobjects 116, which are text-based objects that encode the functionalelements represented by visual inputs 106 into a more-easily handledform. A non-limiting example of such a conversion is detailed below withreference to FIG. 2. Test script generating tool 110 may then usetextual objects 116 with identifier tool 120 to identify the propertiesof textual objects 116 and whether they match with known computer codeelements 136. For example, identifier tool 120 may compare textualobjects 116 with known computer code elements 136 and based on thecomparison, associate certain testing criteria to textual objects 116,e.g., to test for the ability to receive a certain input or to test thatthe navigation goes to the correct destination. Then, test scriptgenerating tool 110 may use script engine 125 to generate test script126. For example, script engine 125 may compile the testing criteria oftextual objects 116 into a series of tests to run as a script, As aresult, system 100 may use test script generating tool 110 to generatetest script 126 from visual inputs 106 from specification document 105.

System 100 may then further test computer programs, such as computercode 146, using the generated test scripts from test script generatingtool 110. The generated test scripts, such as test script 126, may bestored in memory or storage, such as test code repository server 130.These tests scripts may be accessed by automation framework 140 uponrequest, e.g., a script request 141. Automation framework 140 may beconfigured to receive a computer program 146, e.g., from computer coderepository server 145, and run test script 126 or any other suitabletest procedures on computer program code 146. In certain embodiments, adeveloper 150 may upload a computer program developed based onspecification document 105. Automation framework 140 may retrieve thecomputer program code 146 from computer code repository server 145 andtest scripts relevant to specification document 105 from test scriptrepository server 130. Automation framework 140 may then run test script126 against the computer program code 146 and provide feedback, e.g., ina report, indicating any errors corrections needed in the computerprogram code 146. In summary, system 100 generates a test script usingtest script generating tool 110 using visual inputs 106 fromspecification document 105 and tests a developed computer program basedon specification document 105 with automation framework 140 using thegenerated test script.

Specification document 105 used in system 100 may suitably describe thefunctional and other requirements used by developers to write thecomputer code. The development of a computer program or a portion of acomputer program may begin with the drafting of a specification document105. Specification document 105 may include a description of the desiredfunctionality of the computer program to be developed. Specificationdocument 105 may include high-level or abstract depictions of variouselements of the computer program. For example, specification document105 may include pictorial or visual elements that correspond toparticular functional requirements and/or characteristics to be includedin the computer program. Specification document 105 may also includetext that describes the functional requirements, e.g., throughlabels/descriptors of illustrated features or more explicitly insentences describing that portion of the computer program. Specificationdocument 105 may be any document that can be represented and displayedas visual information. For example, specification document 105 may be aWireFrame document, a Control flow diagram, a Business RequirementsDocument (BRD), a UX diagram, a Unified Modeling Language (UMLdocument), etc. Specification document 105 may also be a pdf or other“flat” document from which visual information may be extracted.Accordingly, computer programmer or developer may use specificationdocument 105 or portions thereof as the guidelines for writing computercode.

On the other hand, specification document 105 may also be useful inchecking whether the developed computer program code, e.g., computercode 146, complies with priorities and requirements of developmentproject. For example, a test script may be written that checks whethercertain functional elements described in specification document 105 areincluded in computer code 146. More specifically, if specificationdocument 105 depicts a flow diagram showing a decision based on inputtedinformation, a test script can include a test whether the code includessuch a decision and if it uses that inputted information, As anotherexample, certain user experience styles may be designated byspecification document 105. In such cases, test script 126 may includetests whether code elements comply with one or more style templates fordisplayed elements. As described in further detail below, disclosedherein are certain embodiments of systems and methods that utilize thevisual information of specification document 105 to generate testscripts automatically.

According to certain embodiments, test script generating tool 110 mayobtain visual inputs 106 from specification document and generate a testscript using visual inputs 106. For example, test script generating tool110 may include an interface communicatively coupling test scriptgenerating tool 110 to a server or any other location where visualinputs 106 are stored. Each of visual inputs 106 obtained by test scriptgenerating tool 110 may represent one or more functional requirements ofspecification document 105. For example, one of visual inputs 106 mayindicate the inclusion of a button that navigates a user to a particulardisplay screen that displays specified information in a box according toa style template. A respective one of visual inputs 106 may displayfurther information such as describing potential input criteria foruser-input textbox or what function or functions are called to cause thedisplay of requested information. The level of detail and specificitymay vary based on specification document 105 and/or how visual inputs106 are obtained from specification document. For example, differentfile types may encode information differently such that different visualinput is obtained. As another example, different parsing or protocolsmay result in the loss of metadata or other information when obtainingthe visual data. In this manner, test script generating tool 110 mayobtain visual inputs 106 from specification document 105.

In addition to storing visual inputs 106 in memory, Test scriptgenerating tool 110 may store a set of known computer code elements,such as known computer code elements 136, in the memory. The set ofknown computer code elements may include a set of computer code elementsthat have known characteristics. For example, a respective knowncomputer code element of known computer code elements 136 could be atext box with a first defined style, a navigation button with a seconddefined style, a display box with the first defined style, etc. Each ofthe known computer code elements 136 may have associated with itpredetermined testing criteria for testing computer code that includesthat element. For example, a text box with a first defined style may beassociated with testing criteria such as to test the height and width ofthe displayed text box, the input text allowed, the number of charactersit accepts, and how it displays the text when being inputted. In certainembodiments, the first defined style is associated with a styletemplate, which includes additional or alternative testing criteria tocheck of the element conforms to the style template. This may includetesting for compliance with a color scheme or how element is formattedand displayed to a user. Thereby, test script generating tool 110 mayhave access to known elements 136 to which textual objects 116 generatedfrom visual inputs 106 may be compared.

Test script generating tool 110 need not store all known computer codeelements 136 at all times. In certain embodiments, known computer codeelements 136 may be accessed from known computer code elementsrepository 135 or any other suitable storage location and loaded intothe memory of test script generating tool 110. For example, test scriptgenerating tool 110 may request known elements 136 from known computercode elements repository 135 in response to receive a request togenerate a test script. In some embodiments, the set of known computercode elements 136 requested may depend on specification document 105.For example, specification document 105 may indicate the category of thecomputer program that is being developed and/or what libraries may beused in developing the computer program. In this case, test scriptgenerating tool 110 may only request known computer code elements 136that would be relevant to generating test script 126 based onspecification document 105. In some embodiments, the set of knowncomputer code elements 136 may change over time, e.g., via updates tolibraries, styles, etc. Accordingly, test script generating tool 110 mayperiodically or upon command update the set of known computer codeelements 136 it stores in memory. In some embodiments, test scriptgenerating tool 110 stores the most common sets of known computer codeelements 136 in memory across multiple specification documents, therebyreducing the data usage to/from known computer code elements repository135. In sum, test script generating tool 110 may have access to relevantand complete sets of known computer code elements 136, which are used ingenerating test script 126, as further detailed below.

Conversion engine 115 of test script generating tool 110 uses theplurality of visual inputs 106 to generate corresponding textual objects116. In certain embodiments, conversion engine 115 accesses theplurality of visual inputs 106 from memory and generates a plurality oftextual objects 116 from the plurality of visual inputs 106 using one ormore processors. Converting visual inputs 106 to textual objects 116 mayinclude identifying pictorial elements by classifications (e.g., a shapetype, a control type, dimensions, textual descriptors, etc.) andrendering those classifications into a respective one of textual objects116, such as a string or other text-based data structure. Conversionengine 115 is configured to maintain the association of functionalrequirements of the computer program specification document, e.g.,specification document 105, during the conversion of visual inputs 106and generation of textual objects 116. In this manner, conversion engine115 may convert visual inputs 106 into textual objects that representthe requirements of specification document 105. Textual objects 116 maybe more easily handled by test script generating tool 110 in identifyingthe elements represented by textual objects 116 and generating testscript 126, as described further below. As a result, test scriptgenerating tool 110 may use conversion engine 115 to providemore-easily-handled textual objects 116 for further use in generatingtest script 126 while still maintaining the association with therequirements specified in specification document 105.

Conversion engine 105 may handle a variety of types of visual inputs 106and generate different types of textual objects 116. For example, whilevisual inputs 106 correspond to textual objects 116 generated byconversion engine 115, the mapping between the visual inputs 106 andtextual objects 116 may not be one-to-one. In certain embodiments,conversion engine 115 generates one of textual objects 116 for anindividual one of visual inputs 106, but in other embodiments,conversion engine 115 generates more multiple textual objects 116 for anindividual one of visual inputs 106. In some embodiments, the number oftextual objects 116 generated from an individual one of visual inputs106 may depend on the characteristics of the respective visual input,such as how the respective visual input is delineated from the otherones of visual inputs 106 and in what file type visual inputs 106 areprovided. In this manner, conversion engine 115 may break up visualinputs 105 into more discrete textual objects 116, which are more easilyprocessed by test script generating tool 110. Accordingly, conversionengine 105 may flexibly process visual inputs 106 to generate textualobjects 116 that can be used by other aspects of test script generatingtool 110 to generate a test script.

Test script generating tool 110 may use identifier tool 120 to identifya matching computer code element for each of textual objects 116. Forexample, identifier tool 120 may access the set of known computer codeelements 136 and compare each of textual objects 116 to the set of knownelements 136. In doing so, identifier tool 120 may determine whether therespective one of textual objects 116 matches a respective known elementof known elements 136. Identifier tool 120 may make the comparisonbetween the respective one of textual objects 116 and the set of knownelements 136 by checking if the characteristics of a respective knownelement 136 are present in the respective one of textual objects 116.For example, conversion engine 115 may be configured to convert arespective one of visual inputs 106 into textual objects 116 with acertain format that matches the format of the elements in the set ofknown computer code elements 136, thereby facilitating the comparison todetermine if there is a match far the respective one of textual elements116. In some embodiments, a match is determined if a set of necessarycharacteristics are shared between the respective one of textual objects116 and the matching known element of known elements 136. For example,the respective one of textual objects 116 may include additionalcharacteristics that are not relevant to testing the computer program.Accordingly, a known computer code element of the known elements 136 maynot include the additional characteristics but would still be a desiredmatch because the respective textual object matches all othercharacteristics. In some embodiments, the respective textual object musthave present all of the characteristics of the respective known computercode element 136 to be determined a match. As a result, identifier tool120 may identify, for each of textual objects 116, a match to a knowncomputer code element of the known elements 136, the fact of which maybe further used in determining what testing criteria to consider forgenerated test script 126.

While the comparison described above may be suitable for manyapplications, it may be necessary to include more advanced techniquesfor identifying the respective one of textual objects 116. In certainembodiments, a simple comparison does not yield a match for therespective textual element. In some cases, this is due to the respectivetextual object missing or having unknown a key characteristic foridentifying the respective textual object and what is a correspondingone of the known computer code elements 136. The missing or unknowncharacteristic may be due to any number of causes, such as a lack ofspecificity based on the specification document or having certaincharacteristics defined in separate visual inputs 106.

In certain embodiments, identifier tool 120 may employ additionalcomparison techniques to resolve the missing or unknown aspects of therespective one of textual objects 116. In some embodiments, identifiertool 120 may use a Bayesian comparison to determine a matching elementof known computer code elements 136. For example, the Bayesiancomparison may be trained using machine learning using training setsbased on the set of known computer code elements 136. During theBayesian comparison, identifier tool 120 may the determine the missingattribute of the respective textual element and based on thatdetermination, compare the respective textual element to known elements136 with the added attribute. In some embodiments, the Bayesiancomparison only determines a missing attribute and/or a match with arespective one of known elements 136 if the Bayesian comparison exceedsa match threshold. For example, a threshold of a 90% probability may beset before the Bayesian comparison provides a determination of a missingattribute or a match. In this manner, if there is uncertainty exceeding10% probability, then the Bayesian comparison may return no match. Thethresholds may be determined by a user of script generating tool 110based on an accuracy tolerance of the computer code or end-user oraccuracy of the Bayesian comparison based on its training and priorsuccess rate. In this manner, identifier tool 120 may use a Bayesiancomparison to more accurately identify a matching one of known elements136, even if the respective textual object is missing information usedto determine a match.

In addition to determining whether certain characteristics match or arepresent in the respective one of textual object 116, identifier tool 120may also compare the constituent text of the respective textual objectwith text of known computer code elements 136. In certain embodiments,identifier tool 120 may use a keyword-based sentimental classificationto determine a matching element of known computer code elements 136. Forexample, if the respective textual object is missing a control typeattribute, e.g., button, text box, display text, etc., it may not matchwith any of known computer code elements 136. However, the respectivetextual object may include text that contextually describes its controltype. For example, if the respective textual element includes the text“accepts 120 characters for password” the keyword classification mayassociate the words “accepts 120 characters” with the control attributeof “text box.” Based on the keyword classification, identifier tool 120may associate the determined attribute with the respective textualobject and make the comparison to the set of known computer codeelements 136. Now, with the added attribute, identifier tool 120 mayfind a match to a particular text box element. In this manner,keywork-based sentimental classification may be used by identifier tool120 in comparing textual objects 116 to the set of known computer codeelements 136.

Identifier tool 120 may use any suitable combination of comparison andidentification techniques. In certain embodiments, identifier tool 120is configured to use both the Bayesian comparison and the keywordclassification. In some embodiments, identifier tool 120 employs theBayesian comparison before employing the keyword classification. In someembodiments, identifier tool 120 employs the Bayesian comparison afteremploying the keyword classification. In some embodiments, identifiertool 120 uses the Bayesian comparison only if the keyword classificationfails in finding a match between a respective one of textual objects 116and a respective one of known elements 136. In some embodiments,identifier tool 120 uses the keyword classification only if the Bayesiancomparison fails in finding a match between the respective textualobject and one of known elements 136. In this manner, the comparisontechniques describe above may be combined in a number of different waysto cause identifier tool 120 to most successfully find a match for therespective textual element.

Identifier tool 120 may also be used by test script generating tool 110to determine what tests to include in the generated test script. Forexample, if identifier tool 120 determines a match for a respective oneof textual object 116, it may assign testing criteria associated withthe match to the respective textual object. In certain embodiments, ifthe respective textual object matches one of known elements 136,identifier 120 associates the predetermined testing criteria of therespective one of known elements 136 to the respective textual object.As discussed above, each of known elements 136 may have testing criteriaon which the function of the element may be tested. If the respectivetextual object is a match, then identifier tool 120 associates thecriteria of the matching known element of the known elements 136 to therespective textual object. Identifier tool 120 may repeat this processfor each of textual objects 116 generated based on specificationdocument 105. In this manner, identifier tool 120 identifies whattesting criteria may apply to textual objects 116, and thereby testscript 126.

The testing criteria may include other compliance considerations, suchas the style of the computer code or the style of the resulting userdesign experience. In certain embodiments, the predetermined testingcriteria indicates to apply a style template. For example, a matchingknown element of the known elements 136 may have a characteristic thatindicates a style type or a style template that should be used. In suchcases, identifier tool 120 may access the style template and determineany user experience design requirements associated with the styletemplate. As mentioned above, this may include formatting features,color schemes, and the application of certain logos or backgroundgraphics. One or more style testing criteria may correspond to the oneor more user experience design requirements associated with the styletemplate. Accordingly, identifier tool 120 may associated these styletesting criteria to the respective one of textual objects 116. Asdiscussed below, script engine 125 may generate test script 126 usingthe style testing criteria in addition to any other testing criteriadiscussed herein. Accordingly, identifier fool 120 may associate all ofthe relevant testing criteria to textual objects 116 to be used togenerate test script 126.

Once the testing criteria have been identified and associated withtextual objects 116, identified objects 121 may be sent to script engine125 to generate a test script. Identified objects 121 may include eachtextual object 116 and any testing criteria or related informationassociated with the respective textual object 116. In certainembodiments, script engine 125 is configured to generate a test scriptusing the predetermined testing criteria associated with the pluralityof textual objects 116 as represented as identified objects 121. Forexample, for each testing criteria, script engine 125 may determinewhich tests indicated in the criteria to apply to computer codecorresponding to textual objects 116. Script engine 125 may compile allof the determined tests into a single file or set of instructions. Basedon this file or set of instructions, script engine 125 may generate testscript 126. For example, script engine 125 may generate a json messagecomprising test script 126 for use at automation framework 140. In someembodiments, the generated test script may be generated in a genericsyntax that may be converted for use across a variety of differentautomation frameworks. As a result, test script generating tool 110 mayuse script engine 125 to generate a usable test script to test thecomputer code developed based on specification 105.

Test script generating tool 110 may make test script 126 available foruse by other components of system 100, such as automation framework 140.Once test script 126 is generated, script engine 125 may transmit testscript 126 to another storage location such as test script repositoryserver 130. The storage location may be accessible by automationframework 140. In some embodiments, the transmitted test script alsoincludes identifying information that allows automation framework 140 toidentify from which specification document test script 126 wasgenerated. In this manner, when automation framework 140 is instructedto test a written computer program based on specification document 105,it may retrieve test script 126 generated by test script generating tool110 using visual inputs 106 from specification document 105.

System 100 may cause the generation of test script 126 by tests scriptgenerating tool 110 based on specification document 105 before computerprogram 146 is developed based on specification document 105. Forexample, in certain embodiments, test script generating tool 110generates test script 126 and transmits it test script repository server130 before computer program 146 is written and/or is available fortesting or review. As described above, by using visual inputs 106 ofspecification document 105, test script generating tool 110 may generatetest script 126. Not only does this enable immediate testing once thecomputer program 146 is written, since test scripts 126 are ready to beaccessed and run by automation framework 140 once computer program 146is uploaded to test code repository 145, it may also provide testscripts that are usable with various versions of computer program 146 ifbased on the same specification document 105. In this manner, system 100may provide an improved testing system by providing the generation oftest scripts before the computer program is developed.

System 100 may be configured to allow a user to interact with one ormore components of system 100 to improve and/or provide flexibility tothe generation of test scripts. In certain embodiments, test scriptgenerating tool 110 prompting a user via a user interface for anidentification of a respective one of textual objects 116. For example,if identifier tool 120 cannot determine a match for the respectivetextual object using all available methods, it may request user input tospecify any missing attributes or identifying a matching element. Onceidentified, identifying tool 120 may continue with identifying textualobjects 116 and associating respective testing criteria, as detailedabove. In some embodiments, this user input may be used to train thecomparison techniques used by identifier tool 120, thereby continuing toimprove the accuracy of the comparison and reduce the amount of userinput needed to generate test script 126. In this manner, system 100 mayuse user input to improve the identification of textual objects 116.

User interaction with code testing system 100 may also be used toprovide flexibility in test scripts 126 generated by test scriptgenerating tool 110. In certain embodiments, a user provides one or moreinstructions to test script generating tool 110. Based on thoseinstructions, test script generating tool 110, may determine a subset ofthe predetermined testing criteria for use in generating test script126. For example, before generating test script 126 script engine 125may prompt the user to choose which predetermined testing criteria toapply. The user may choose all, some, or none to apply. The user maydetermine that certain tests are redundant, unnecessary for theapplication of computer program 146 or would require too much time torun. In such cases, the use may unselect or otherwise indicate toexclude those tests. Once the subset is submitted, script engine 125 maygenerate test script 126 that reflects the user's input. As a result,test script generating tool 110 may provide flexibility in auser-friendly way by expanding user choice and influence in generatingtest script 126.

As discussed earlier, after test script 126 is generated by test scriptgenerating tool 110, test script 126 may be sent to test scriptrepository 130. Test script repository 130 may include storage in whichtest script 126 may be stored. In certain embodiments, each of testscripts 126 received by test script repository 130 are associated withthe specification document that provided visual inputs 106 to testscript generating tool 110. For example, test script generating tool 110may provide an indicator identifying the relevant specification documentwhen transmitting a test script to test script repository 130. Testscript repository 130 may store the indicator together with test script126 so that the relevant test scripts may be accessed based on whatspecification document was used. For example, automation framework 140may request in a script request 141, e.g., an access request, one ormore test scripts from test script repository 130 to use to test acompiled computer program. Automation framework 140 may identify thespecification document used in writing the compiled computer program inscript request 141 to test script repository 130 for relevant testscripts. Test script repository 130 may use the provided identificationto retrieve the relevant test scripts and send test script 126 toautomation framework 140. In this manner, test script repository 130 mayprovide storage for test scripts generated by test script generatingtool 110 to be used by automation framework 340 for testing compiledcomputer program code 146.

System 100 is described above as using automation framework 140 togetherwith test script generating tool 110 to test computer program code 146.Automation framework 140 may include any automation framework that runstest scripts to test computer program code 146. For example, automationframework 140 may be configured to receive computer program code 146,e.g., from code repository 145, and run one or more test scripts againstcomputer program code 146. In some embodiments, automation framework 140is integrated with test script generating tool 110. For example,automation framework 140 and test script generating tool 110 may sharehardware processing and/or memory resources and/or software resources.In this manner, automation framework ma be any suitable automationframework as understood by a person having ordinary skill in the art.

In certain embodiments, automation framework 140 may generate a reportindicating any passed and/or failed tests completed using the one ormore test scripts. The report may identify the location within computercode 146 of a failed test. The report may also identify what test wasfailed. In this manner, the developer may be provided feedback whetherhis code complied with the specification document. If the developer'scode does not comply, e.g., it failed one or more tests carried outusing test script 126, the report may provide information sufficient forthe developer to fix the code so that it complies with the specificationdocument. As a result, automation framework 140 may provide valuablefeedback that can be used to develop more robust and compliant computerprograms.

As discussed above, code repository server 145 may store computerprograms, such as computer code 146, that have been developed and areready for testing, e.g., via automation framework 140. For example, thedeveloper may use specification document 105 to write computer program146. The developer may compile computer program 146 and upload it tocode repository server 145. Code repository server 145 may be accessedby automation framework 140 to retrieve compiled computer program 146 tobe tested. In certain embodiments, code repository 145 sends scriptrequest 141 to automation framework 140 to test compiled computerprogram 146. In some embodiments, code repository 145 sends scriptrequest 141 as an automatic response to compiled computer program beinguploaded to code repository 145. In some embodiments, the developer mayindicate to code repository 145 to request automation framework to testthe compiled computer program when uploading the compiled computerprogram to code repository 145. Upon the request, automation framework140 may queue the compiled computer program for testing using the one ormore of the generated tests scripts by test script generating tool 110.Code repository server 145 may be implemented in any suitablehardware/software combination. For example, code repository server 145may include any suitable processors, memory, and interfaces as readilyunderstood in the art. In this manner, code repository server 145 may beused to store relevant computer program code for testing in system 100.

In certain embodiments, code testing system 100 may further includeoptional machine learning training server 155. Machine learning trainingserver 155 may be configured to train comparison algorithms or schemesused by identifier 120, such as the Bayesian comparison and thekeyword-based sentimental classifier described herein. For example, adeveloper or compliance manager may input training cases to machinelearning trainer server 155 that correspond to correct matches betweentextual objects 116 and known computer code elements 136. Using theseinput training cases, Machine learning training server 155 may adjustone or more comparison parameters 156 used in smart comparisons that maybe used to match textual objects 116 and known elements 136. Forexample, the developer may provide a set of textual elements 116 thathave been manually identified to correspond to certain known computercode elements 136 to Machine learning training server 155. Machinelearning training server 155 may attempt to identify textual elements116 using the existing comparison parameters 156. Machine learningtraining server 155 may then check to see if it arrived at the correctsolution. If Machine learning training server 155 arrived at a differentsolution or did not arrive at any solution, Machine learning trainingserver 155 may adjust one or more comparison parameters 156 based on thedifferences in the solutions and/or the correct solution. If Machinelearning training server 155 provides the correct solution, it mayreinforce the most relevant of comparison parameters 156. Machinelearning training server 155 may perform these adjustment for the all ofthe test cases provided by the developer.

Once trained, the updated comparison parameters 156 may be used with thesmart comparisons, such as the Bayesian comparison and the keyword-basedsentimental classification. This allows for new known elements to beadded to known objects 135 for use in identifying textual objects 116.Further it increases the accuracy for identifying a respective matchingknown object of the known objects 136. As a result, the resulting testscripts generated by test script generating tool 110 may improve overtime and machine learning iterations, leading to improved testing andless buggy computer programs. In this manner, certain machine learningaspects may be integrated into test script generating tool 110 toimprove its accuracy and reduce the amount of user input required togenerate test scripts.

While only a brief discussion of machine learning is provided herein, itis understood to those having skill in the art that there are amultitude of machine learning training and reinforcement methods andsystems that may be used with code testing system 100. Any suitablemachine training apparatus or methods are contemplated for use in thevarious embodiments disclosed herein.

FIG. 2 illustrates example visual input 106 a and textual objects 116 aused in generating a test script, e.g., using test script generatingtool 110. Visual input 106 a and textual objects 116 illustratenon-limiting examples of visual input 106 and textual objects 116, butother suitable visual inputs 106 and textual objects 116 arecontemplated herein. Visual input 106 a may be visual input obtainedfrom specification document 105 that includes user design interfaces.For example, visual input 106 a may include elements that correspond topictorial elements (e g., arrows, wireframe buttons, decision elements,etc.) and text elements (e.g., names, descriptors of elements, textdescribing functionality, yes/no forks, etc.). In the illustratedexample, visual input 106 a includes two display screens that areconnected by an arrow. The left display screen is labeled “Home Screen”and includes two dashed boxes labeled “UserName” and “Password.”respectively, and a solid-lined oval labeled “Submit.” The right displayscreen is labeled “Dashboard” and includes two solid-lined boxes labeled“Customer Name” and “Account Balance,” respectively and a solid-linedoval labeled “Contact Us.” The arrow shows a direction from the leftscreen to the right screen. Visual input 106 a may be one of theplurality of visual inputs 106 that is obtained by test scriptgenerating tool 110 and converted by conversion engine 115.

Conversion engine 115 may generate textual objects 116 a based on visualdata 205. As in this illustrated example, textual objects 116 a mayinclude a string of text that describe or represent visual input 106 a.For example, the top one of textual objects 116 a corresponds to theleft screen “Home Screen” and the bottom on of textual objects 116 acorresponds to the right screen “Dashboard.” As provided in thisexample, textual objects 116 a encode the functional information formthe specification document displayed in visual input 106 a in a textform. In particular, textual objects 116 a encode the name of the screenas “PageName” and describe each element present on the screen. Forexample, the top one of textual objects 116 a includes four elements,elements 1, 2, 3, and 4. Element 1 describes the “UserName” box by itsname, the fact it is depicted by a dashed-lined box, and its dimensions.Element 2 describes the “Password” box by the same features. Element 3describes the “Submit” button by its name, the fact it is, an ovalbutton, and its dimensions. Lastly, Element 4 describes the fact thereis an arrow between the screens and it extends from “Home Screen” to“Dashboard.” The bottom one of textual objects 116 a encodes similarinformation except for the different text descriptors used and the factthat the boxes are not dashed boxes, but solid-lined boxes. Accordingly,textual objects 116 a encode visual input 106 a for use by identifiertool 120 by providing a text-based form for describing the functionalelements of the specification document that may be tested using a testscript.

While FIG. 2 provides an example of the type of visual input 106 a andtextual objects 116 a that may be used with test script generating tool110, any suitable visual input and textual objects may be used inaccordance with the disclosed techniques of generating a test script.For example, visual input 106 a may take different forms based onspecification document 105, e.g., based on the file type ofspecification document 105 or what methods are used to parse visualinput 106 a from specification document 105. As a specific example,scanned images from a printed version of specification document 105 mayresult in very different visual input 106 a than from parsing a Visiospecification document. Likewise textual objects 116 a may differ basedon the type of visual input 106 a obtained by test script generatingtool 110 and/or the implementation of identifier tool 120 and/or scriptengine 125. For example, different representations of textual objects116 a may be more useful (or compatible) with different programminglanguages and/or subroutines used by test script generating tool 110. Inthis manner, visual input 106 a and textual objects 116 a is not limitedto the disclosed example embodiments, and may take any suitable form asunderstood by a person having skill in the art.

FIG. 3 is a flowchart of an embodiment of a test script generatingmethod 300 for use in an automation framework. Test script generatingtool 110 may employ method 300 for generating a test script based onvisual inputs 106 from specification document 105.

At step 305, a plurality of visual inputs 100 is obtained from acomputer program specification document 105. Each respective one ofvisual inputs 106 represents one or more functional requirements ofspecification document 105. For example, test script generating tool 110may obtain visual inputs 106 from a server storing specificationdocument 105.

At step 310, a plurality of textual objects 116 is generated from theplurality of visual inputs 106. For example, conversion engine 115 oftest script generating tool 110 may convert a respective one of visualinputs 106 into one or more textual objects 116 that represent therespective one of visual inputs 106 and the underlying functionalrequirements from specification document 105. In this manner, textualobjects 116 maintain an association with the requirements ofspecification document 105 but are more easily handled by identifiertool 120 and script engine 125 because they are textual objects insteadof visual data.

At step 315, a set of known computer code elements 136 is accessed bytest script generating tool 110. Each respective element of the set ofknown computer code elements 136 includes predetermined testing criteriafor testing computer code that includes the respective element. Forexample, a respective one of known computer code elements 136 mayinclude a page navigation button. The page navigation button may havepredetermined testing criteria such as instructions to test thedimensions and color of the page navigation button or to test that thebutton is clickable and that it navigates to the correct page. The setof known computer code elements 136 may be stored in a separaterepository or server or may be stored with test script generating tool110. In some embodiments, the set of known computer code elements 136may be updated and test script generating tool 110 may re-access the setof known code elements 136 to use the latest version.

Using the set of known computer code elements 136, at step 320, theplurality of textual objects is compared to determine whether any oftextual objects 116 matches one or more of known elements 136. Forexample, a respective textual object of textual objects 116 generatedfrom a respective visual input of visual inputs 106 corresponding to atextbox may match a known textbox element. The comparison may check ifthe textual object matches all of the characteristics of the respectiveone of known elements 136. In some embodiments, more advancedcomparisons may be made. For example, machine learning trainedalgorithms employing a Bayesian comparison or a keyword classificationanalysis may be employed by identifier tool 120. In particular, aBayesian comparison may be used to check if a respective one of textualobjects 116 matches a known element even if the respective one textualobject is not exactly the same as the respective one of known elements136, e.g., is missing certain characteristics or the characteristics areslightly different. Also, the keyword classification may be used tocheck if the respective textual object matches one of known elements 136using contextual text information from the respective visual input usedto create the respective textual object.

Based on the comparison in step 320, at step 325, identifier tool 120determines if the respective one of textual objects 116 matches one ofknown elements 136. If it does not match, method 300 may proceed to step330, wherein additional information from another source is requested.For example, test script generating tool 110 may request input from user150 to identify one of textual objects 116 and/or visual inputs 106 fromwhich it was converted. As another example, test script generating tool110 may request additional known elements. In such cases, method 300 mayreturn to step 320 to further compare the respective textual object tothe additional sets of known elements. In some embodiments, this processis repeated until a known element is matched with the respective textualobject. In some embodiments, the respective textual object may not havea match to one of known elements 136.

If, on the other hand, the respective one of textual object matches aknown computer code clement 136, method 300 may proceed to step 335,wherein testing criteria of the respective one of known elements 136 isassociated with the respective textual object. For example, if therespective textual object matches with a display box, the respectivetextual object may be associated with testing criteria including testingfor the height and width of the displayed box and testing for thecontent displayed in the displayed box. As a result, script engine 125may know what tests to include in test script 126 for each of textualobjects 116 based on the testing criteria associated with eachrespective one of textual objects 116.

In certain embodiments, steps 320, 325, 330, and/or 335 may be repeatedfor each textual object of textual objects 116 converted from visualinputs 106 obtained by script generating tool 110. In this manner, eachof textual objects 116 may be analyzed to determine all appropriatetesting criteria to include in test script 126 to be generated.

At step 340, a test script 126 is generated using the predeterminedtesting criteria associated with textual objects 116. For example,script engine 125 of test script generating tool 110 may receiveidentified objects 121 containing identified textual objects 116 and/ortheir associated testing criteria and parse the testing criteria intotests. The tests may be compiled into a script suitable for being ran byan automation tool such as automation framework 140. As a result, testscript 126 may be generated by test script generating tool 110 usingvisual inputs 106 from specification document 105.

At step 345, test script 126 is transmitted to a test script repositoryserver, such as test script repository sever 130. For example, aftertest script generating tool 110 generates test script 126 it may connectwith a server on which test scripts are stored. The tests scriptgenerating tool 110 may also include identifying information with testscript 126, separately or embedded in test script 126. For example,before transmission, test script generating tool 110 may associate testscript 126 with the specification document 105 and/or identifyinginformation from specification document 105. In certain embodiments,automation framework 140 may access test script 126 from test scriptrepository server 130 for use in testing computer program 146 writtenbased on specification document 105. As a result of method 300, testscript 126 is generated and made accessible based on the plurality ofvisual inputs 106 is obtained from computer program specificationdocument 105.

In certain embodiments, method 300 may include more or fewer stepsand/or one or more substeps. For example, in certain embodiments, method300 further includes receiving instructions from and/or data from auser. The received instructions and/or data may be used in determiningwhich one of known elements 136 match respective ones of textual objects116 and/or what testing criteria to include in the generated testscript. Additionally, a user may instruct test script generating tool110 to start or stop the generation of a test script. For example, auser may send a request to generate test script 126 based onspecification document 105 that the user also specifies and providesaccess to. Accordingly, user input may be incorporated to ensure theaccuracy of test script 126 and provide flexibility in what test script126 tests.

While certain components of test script generating tool 110 aredescribed as carrying out one or more steps of method 300, any suitablecomponent of code testing system 100 may be used to carry out thevarious steps of the methods described herein.

FIG. 4 is a schematic diagram of an example test script generating tool410 configured to generate test scripts using visual inputs, such astest script 126 from visual inputs 106. Test script generating tool 410includes an interface 430, a memory 435, and a processor 440. Testscript generating tool 410 may be configured in the illustratedconfiguration or in any other suitable configuration.

Interface 430 is configured to enable wired and/or wirelesscommunications. Interface 430 is configured to communicate data betweenelements of code testing system 100, such as test script repository 130,known code elements repository 135, machine learning training server155, and a server containing specification document 105, or between testscript generating tool 410 and a remote system. For example, interface430 may include a WIFI interface, a local area network (LAN) interface,a wide area network (WAN) interface, a modem, a switch, or a router.Processor 440 may be configured to send and receive data using theinterface 430. Interface 430 may be configured to use any suitable typeof communication protocol as would be appreciated by one of ordinaryskill in the art.

Memory 435 comprises one or more disks, tape drives, or solid-statedrives, and may be used as an over-flow data storage device, to storeprograms when such programs are selected for execution, and to storeinstructions and data that are read during program execution. Memory 435may be volatile or non-volatile and may comprise, read-only memory(ROM), random-access memory (RAM), ternary content-addressable memory(TCAM), dynamic random-access memory (DRAM), and static random-accessmemory (SRAM). Memory 435 is operable to store instructions forconverting visual inputs into textual objects, instructions foridentifying textual objects and applying testing criteria, andinstructions for generating a test script based on the testing criteria,and/or any other data, or instructions that may be used to carry out thefunctions and processing described herein. The instructions in memory435 may include any suitable set of instructions, logic, rules, or codeoperable to execute conversion engine 415, identifier tool 420, andscript engine 425. Additionally, memory 435 may be operable to storeother information used or generated by conversion engine 415, identifiertool 420, or script engine 425. For example, memory 435 may beconfigured to store obtained visual inputs 106, textual objects 116created by conversion engine 415, a set of known computer code elements136 accessed from code elements repository 135, machine learning trainedcomparison parameters 156, the associations of testing criteria totextual objects 116, and the generated test script by script engine 425,e.g., test script 126.

Processor 440 comprises one or more processors operably coupled to thememory 435 and/or interface 430. Processor 440 is any electroniccircuitry including, but not limited to, state machines, one or morecentral processing unit (CPU) chips, logic units, cores (e.g. amulti-core processor), field-programmable gate array (FPGAs),application specific integrated circuits (ASICs), or digital signalprocessors (DSPs). Processor 440 may be a programmable logic device, amicrocontroller, a microprocessor, or any suitable combination of thepreceding. Processor 440 is communicatively coupled to and in signalcommunication with the memory 435 and/or interface 430. The one or moreprocessors are configured to process data and may be implemented inhardware or software. For example, processor 440 may be 8-bit, 16-bit,32-bit, 64-bit or of any other suitable architecture. Processor 440 mayinclude an arithmetic logic unit (ALU) for performing arithmetic andlogic operations, processor registers that supply operands to the ALUand store the results of ALU operations, and a control unit that fetchesinstructions from memory and executes them by directing the coordinatedoperations of the ALU, registers and other components.

The one or more processors are configured to implement variousinstructions. For example, the one or more processors are configured toexecute instructions to implement conversion engine 415, identifier tool420, and script engine 425. In this way, processor 440 may be a specialpurpose computer designed to implement function disclosed herein. In anembodiment, conversion engine 415, identifier tool 420, or script engine425 are each implemented using logic units, FPGAs, ASICs, DSPs, or anyother suitable hardware.

Conversion engine 415, identifier tool 420, or script engine 425 mayimplement the various feature of conversion engine 115, identifier tool120, or script engine 125 as described in FIG. 1, respectively.

While not separate illustrated, each of test script repository 130,known elements server 135, automation framework 140, code repositoryserver 145, and machine learning training server 155 may be implementedusing any suitable computer hardware, software, or combination thereof.For example, each of test script repository 130, known elements server135, automation framework 140, code repository server 145, and machinelearning training server 155 may include one or more of a processor, amemory, and an interface, as described above in implementing the exampletest script generating tool 410. In certain embodiments, certaincomponents of system 100 may be integrated together and/or share certainhardware and/or software resources. This disclosure includes anysuitable configuration of the components and/or functionality describedherein.

While several embodiments have been provided in the present disclosure,it should be understood that the disclosed systems and methods might beembodied in many other specific forms without departing from the spiritor scope of the present disclosure. The present examples are to beconsidered as illustrative and not restrictive, and the intention is notto be limited to the details given herein. For example, the variouselements or components may be combined or integrated in another systemor certain features may be omitted, or not implemented.

In addition, techniques, systems, subsystems, and methods described andillustrated in the various embodiments as discrete or separate may becombined or integrated with other systems, modules, techniques, ormethods without departing from the scope of the present disclosure.Other items shown or discussed as coupled or directly coupled orcommunicating with each other may be indirectly coupled or communicatingthrough some interface, device, or intermediate component whetherelectrically, mechanically, or otherwise. Other examples of changes,substitutions, and alterations are ascertainable by one, skilled in theart and could be made without departing from the spirit and scopedisclosed herein.

To aid the Patent Office, and any readers of any patent issued on thisapplication in interpreting the claims appended hereto, applicants notethat they do not intend any of the appended claims to invoke 35U.S.C. §112(f) as it exists on the date of filing hereof unless the words “meansfor” or “step for” are explicitly used in the particular claim.

The invention claimed is:
 1. A test script generating tool forgenerating a test script for use in an automation framework, comprising:an interface communicatively coupling the test script generating tool toa server, wherein the test script generating tool accesses the servervia the interface to obtain a plurality of visual inputs from a computerprogram specification document, wherein each visual input represents oneor more functional requirements of the computer program specificationdocument a memory storing the plurality of visual inputs and a set ofknown computer code elements, wherein each respective element of the setof known computer code elements comprises predetermined testing criteriafor testing computer code that includes the respective element; aconversion engine configured to: access the plurality of visual inputsfrom the memory; and generate a plurality of textual objects from theplurality of visual inputs using one or more processors, wherein theplurality of textual objects is associated with the one or morefunctional requirements of the computer program specification document;an identifier tool configured to: access the set of known computer codeelements from the memory; compare the plurality of textual objects tothe set of known computer code elements to determine, for eachrespective textual object of the plurality of textual objects, whetherthe respective textual object matches a respective element of the set ofknown computer code elements; and if the respective textual objectmatches a respective element of the set of known computer code elements,associate the predetermined testing criteria of the respective elementto the respective textual object; and a script engine configured to:generate a test script using the predetermined testing criteriaassociated with the plurality of the textual objects; and transmit thetest script to a test script repository server, wherein the automationframework may access the lest script from the test script repositoryserver.
 2. The test script generating tool of claim 1, wherein the testscript is generated and transmitted to the test script repository serverprior to a compilation of a computer program written based on thecomputer program specification document.
 3. The test script generatingtool of claim 1, wherein: the predetermined testing criteria indicatesto apply a style template; associating the predetermined testingcriteria of the respective element to the respective textual objectcomprises: determining one or more user experience design requirementsassociated with the style template; and associating one or more styletesting criteria to the textual object, wherein the one or more styletesting criteria correspond to the one or more user experience designrequirements associated with the style template; and the script enginegenerates the test script using at least the one or more style testingcriteria associated with the respective textual object.
 4. The testscript generating tool of claim 1, wherein comparing the plurality oftextual objects to the set of known computer code elements comprisesapplying a machine learning trained Bayesian comparison to therespective textual object against the set of known computer codeelements to determine whether the respective textual object matches arespective element of the set of known computer code elements.
 5. Thetest script generating tool of claim 1, wherein comparing the pluralityof textual objects to the set of known computer code elements comprisesapplying a machine learning trained keyword-based classification to therespective textual object against the set of known computer codeelements to determine whether the respective textual object matches arespective element of the set of known computer code elements.
 6. Thetest script generating tool of claim 1, wherein: the test scriptgenerating tool is communicatively coupled to a user interface; andcomparing the plurality of textual objects to the set of known computercode elements comprises: determining that the respective textual objectdoes not match any element in the set of known computer code elements;prompting a user via the user interface for an identification of thetextual object; and based on the identification, associating testingcriteria to the respective textual object.
 7. The test script generatingtool of claim 1, wherein: the test script generating tool iscommunicatively coupled to a user interface enabling a user to provideone or more instructions to the test script generating tool; and thescript engine generating the test script comprises determining a subsetof the predetermined testing criteria for use in generating the testscript based on the one or more instructions.
 8. A method for generatinga test script for use in automation framework, comprising: obtaining aplurality of visual inputs from a computer program specificationdocument, wherein each visual input represents one or more functionalrequirements of the computer program specification document; generatinga plurality of textual objects from the plurality of visual inputs,wherein the plurality of textual objects is associated with the one ormore functional requirements of the computer program specificationdocument; accessing a set of known computer code elements, wherein eachrespective element of the set of known computer code elements comprisespredetermined testing criteria for testing computer code that includesthe respective elements; comparing the plurality of textual objects tothe set of known computer code elements to determine, for eachrespective textual object of the plurality of textual objects, whetherthe respective textual object matches a respective element of the set ofknown computer code elements; if the respective textual objet matches arespective element of the set of known computer code elements,associating the predetermined testing criteria of the respective elementto the respective textual object; generating a test script using thepredetermined testing criteria associated with the plurality of thetextual objects; and transmitting the test script to a test scriptrepository server, wherein the automation framework may access the testscript from the test script repository server.
 9. The method of claim 8,wherein generating the test script and transmitting the test scriptoccur prior to a compilation of a computer program written based an thecomputer program specification document.
 10. The method of claim 8,wherein: the predetermined testing criteria indicates to apply a styletemplate; associating the predetermined testing criteria of therespective element to the respective textual object comprises:determining one or more user experience design requirements associatedwith the style template; and associating one or more style testingcriteria to the textual object, wherein the one or more style testingcriteria correspond to-the one or more user experience designrequirements associated with the style template; and generating the testscript comprises using at least the one or more style testing criteriaassociated with the respective textual object.
 11. The method of claim8, wherein comparing the plurality of textual objects to the set ofknown computer code elements comprises applying a machine learningtrained Bayesian comparison to the respective textual object against theset of known computer code elements to determine whether the respectivetextual object matches a respective element of the set of known computercode elements.
 12. The method of claim 8, wherein comparing theplurality of textual objects to the set of known computer code elementscomprises applying a machine learning trained keyword-basedclassification to the respective textual object against the set of knowncomputer code elements to determine whether the respective textualobject matches a respective element of the set of known computer codeelements.
 13. The method of claim 8, wherein comparing the plurality oftextual objects to the set of known computer code elements comprises:determining that the respective textual object does not match anyelement in the set of known computer code elements; prompting a user viaa user interface for an identification of the textual object; and basedon the identification, associate predetermined testing criteria to therespective textual object.
 14. The method of claim 8, further comprisingreceiving one or more instructions from a user via a user interface;wherein generating the test script comprises determining, based on theone or more instructions, a subset of the predetermined testing criteriafor use in generating the test script.
 15. A system for testing computercode, comprising: an automation framework configured to run test scriptsto test computer code programs; a test script generating tool forgenerating a test script for use in the automation framework; a testscript repository server communicatively coupled to the test scriptgenerating tool, wherein the test script repository comprises a memoryfor storing the test script generated by the test script generatingtool; and a computer code repository server communicatively coupled tothe test automation framework, wherein the computer code repositorycomprises a memory for storing a compiled computer code; wherein thetest script generating tool comprises: an interface communicativelycoupling the test script generating tool to a server, wherein the testscript generating tool accesses the server via the interface to obtain aplurality of visual inputs from a computer program specificationdocument, wherein each visual input represents one or more functionalrequirements of the computer program specification document a memorystoring the plurality of visual inputs and a set of known computer codeelements, wherein each respective element of the set of known computercode elements comprises predetermined testing criteria for testingcomputer code that includes the respective element; a conversion engineconfigured to: access the plurality of visual inputs from the memory;and generate a plurality of textual objects from the plurality of visualinputs using one or more processors, wherein the plurality of textualobjects is associated with the one or more functional requirements ofthe computer program specification document; an identifier toolconfigured to: access the set of known computer code elements from thememory; compare the plurality of textual objects to the set of knowncomputer code elements to determine, for each respective textual objectof the plurality of textual objects, whether the respective textual ofmatches a respective element of the set of known computer code elements;and if the respective textual object matches a respective element of theset of known computer code elements, associate the predetermined testingcriteria of the respective element to the respective textual object; anda script engine configured to: generate a test script using thepredetermined testing criteria associated with the plurality of thetextual objects; and transmit the test script to the test scriptrepository server, wherein the automation framework may access the testscript from the test script repository server; and wherein theautomation framework is configured to: access the test script from thetest script repository server and access the compiled computer programfrom the computer code repository server; and run the test script on thecompiled computer program.
 16. The system for testing computer code ofclaim 15, further comprising a machine learning training servercommunicatively coupled to the test script generating tool, wherein thetest script generating tool comparing the plurality of textual objectsto the set of known computer code elements comprises applying a machinelearning trained comparison trained by the machine learning trainingserver.
 17. The system for testing computer code of claim 15, wherein:the test the test script generating tool is communicatively coupled to auser interface; and comparing the plurality of textual objects to theset of known computer code elements comprises: determining that therespective textual object does not match any element in the set of knowncomputer code elements; prompting a user via the user interface for anidentification of the textual object; and based on the identification,associating testing criteria to the respective textual object.
 18. Thesystem for testing computer code of claim 15, wherein: the test scriptgenerating tool is communicatively coupled to a user interface enablinga user to provide one or more instructions to the test script generatingtool; and the script engine generating the test script comprisesdetermining a subset of the predetermined testing criteria for use ingenerating the test script based on the one or more instructions. 19.The system for testing computer code of claim 15, wherein the automationframework is configured to automatically access the compiled computerprogram from the computer code repository server in response toreceiving a notification from the computer code repository server thatthe compiled computer program has been uploaded to the computer coderepository server.
 20. The system for testing computer code of claim 15,wherein the automation framework is configured to automatically accessthe test script from the test script repository server by sending anaccess request to the test script repository server, wherein the accessrequest is used to identify the test script associated with the compiledcomputer program.